Mirror pod environmental sensor arrangement for autonomous vehicle enabling compensation for uneven road camber

ABSTRACT

An approach to arrange sensors needed for automated driving, especially where semitrailer trucks are operating in an autonomous convoy with one automated or semi-automated truck following another. The sensors are fitted to a location adjacent to or within the exterior rearview mirrors, on each of the left- and right-hand side of the tractor. The sensors provide overlapping fields of view looking forward of the vehicle and to both the left and right hand sides at the same time.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 17/476,574 filed Sep. 16, 2021, entitled “MIRROR PODENVIRONMENTAL SENSOR ARRANGEMENT FOR AUTONOMOUS VEHICLE ENABLING LANECENTER OFFSET MIMICRY which is a continuation of U.S. patent applicationSer. No. 16/899,669 filed Jun. 12, 2020 entitled “MIRROR PODENVIRONMENTAL SENSOR ARRANGEMENT FOR AUTONOMOUS VEHICLE” which claimspriority to a U.S. Provisional Patent Application Ser. No. 62/861,502filed Jun. 14, 2019 entitled “MIRROR POD ENVIRONMENTAL SENSORARRANGEMENT FOR AUTONOMOUS VEHICLE” the entire contents of each of whichare hereby incorporated by reference.

BACKGROUND

Proper operation of autonomous vehicles is heavily reliant on camerasand other sensors to detect the presence of nearby objects and otheroperating conditions. One common approach for cars is to mount camerasand other sensors as close as possible to the vehicle's plane ofsymmetry. However, sensor placement in other types of vehicles, such assemitrailer trucks, have different considerations.

International patent application WO2017196165A1 (DAF Trucks NV) entitled“Platooning method for application in heavy trucks” shows side viewmirrors that include lane marking detectors and forward looking cameras.As a result, a reference point P at the back of the trailer of theleading vehicle, can be used to reduce the headway distance attainablewith a single, center-mounted camera. It is also said that, since thelane detector is mounted outside the vehicle width, at least one of thedevices is always able to measure the relative position, relativeheading and curve radius of the current lane.

U.S. Patent Publication US2018/0372875A1 (Uber Technologies) shows asensor assembly that includes replacement side mirrors.

European Patent Publication EP3138736B1 (MAN Truck and Bus AG) alsoshows a mirror replacement system that includes cameras.

SUMMARY

In the case of automated semitrailer trucks, using a high mounted centerposition on the truck is not ideal, since the long, extended hood of thecab can block the forward view of items such as lane markings and otherobjects. Also, when platooning two trucks, the closer a follower vehicleis to the leader, the closer the camera on the follower is to thetrailer of the leader, and thus the less the camera on the follower cansee of its surroundings.

The approach described here places a suite of sensors (e.g., lidardetectors and forward- and rear-facing cameras) in a location adjacentto or within the exterior rear view mirror housings on both sides of thecab. The lidar positioning arrangement minimizes interference andenhances sensor coverage. For example, these mounting positions enable apair of lidars to cover both the peripheral and blind spots areas on thesides of the truck, and to also cover the area in front occupied by alead vehicle at the same time. In addition, forward-facing cameras cancapture images of the rear of the lead vehicle, as well as the road downeither side of the vehicle, important for tasks such as lane followingand obstacle avoidance. The rear-facing cameras can be angled to cover afield of view down the side of the truck and trailer and a large area toeither side.

The sensors can be placed in a separate enclosure mounted to an existingmetal casting of standard rearview truck mirrors. Other implementationscan include custom integrated assemblies specific to each truck model.Cabling for the sensors are routed internally through or along themirror mounting arm into the truck door cavity, which may be via aflexible door hinge to interface with the main autonomy electronics.

In one particular arrangement, a pair of assemblies is provided withinor on both sides of a vehicle such as an autonomous or semi-autonomoustruck. Each assembly includes two or more perception sensors which arepositioned and oriented relative to each other so as to have asubstantially non-overlapping field of view (FOV). The perceptionsensors in each assembly typically include a forward facing sensor and arearward facing sensor that have at least one region of overlappingfield of view alongside the truck. The perception sensors are furtherdisposed such that both (i) the lane markings adjacent to the truck, and(ii) lane markings adjacent at least one truck forward or behind it arealso within a field of view of at least one perception sensor. As aresult, vehicles forward, behind and to the side of the autonomous truckare always within a field of view of at least one perception sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example semitrailer truck having a suite of sensors asdescribed herein.

FIGS. 2A and 2B show the sensors in a typical location within oradjacent a rear view mirror.

FIGS. 3A to 3D illustrate preferred arrangements for the sensors' fieldof view.

FIGS. 3E and 3F further illustrate sensor fields of view with respect toa road that has nonzero camber in the longitudinal and lateraldirection, respectively.

FIG. 4 shows how a pair of sensors detect lane markings on either sideof the truck including any object in a blind spot.

FIG. 5A is an example of what the right-side, rear-facing camera sees ina particular situation with a vehicle in the adjacent lane.

FIG. 5B shows detected center lane offset.

FIG. 5C shows an ideal center lane offset of zero.

FIG. 5D is a flowchart describing visual measurement of such lane centeroffset of the follower vehicle.

FIG. 6A is an example view from the forward-facing camera of a followervehicle.

FIG. 6B depicts a leader center lane offset.

FIG. 6C is a flowchart depicting measurement of lane center offset ofthe leader.

FIG. 7 shows a set of lidar data points detected on the right side of afollower.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 shows the front of a typical vehicle such as semitrailer truck100. A tractor 110 (typically having a cab portion in which a driversits) is towing a trailer 120. Of particular interest is a way topackage and manage the sensors 300 needed for automated orsemi-automated driving, especially where two (or more) trucks 100 aretravelling in a group or platoon, with one automated or semi-automatedtruck following another as companions. Here, the sensors 300 are fittedto a location adjacent to or within each of two exterior rearviewmirrors 200-L, 200-R, located on a respective left- and right-hand sideof the cab 110. As described in more detail below, the sensors 300 arearranged to provide overlapping fields of view looking forward of thetruck 100 and to both the left and right hand sides of the truck 100 atthe same time.

FIGS. 2A and 2B show one way in which these sensors 300 may be packaged,potentially integrated with other components. Here the familiar rearview, flat glass mirror 210 (sometimes referred to as a “West Coast”mirror in the United States), is seen with another glass convex mirror220 disposed below it. Above these optical mirrors 210, 220 are mountedsensors 300, such as a pair of lidar sensors 310, 320 and a pair ofdigital video cameras 330, 340. It should be understood that in otherarrangements, other lidars or cameras, radars, sonars can be included,or indeed, any perception sensor 300 that detects a physical property ofa nearby object or some other condition related to the area adjacent thetruck 100.

The upper most sensor 310 is a blind spot lidar, which may for example,be mounted with a 16-degree pitch and 45-degree roll orientation (pitchbeing referenced to a center lateral axis of the vehicle 100 and rollwith reference to a center longitudinal axis of the vehicle 100). Theblind spot lidar is primarily responsible for looking primarilybackwards from the mirror position. A second, peripheral lidar,functions to detect objects forward of the mirror and to the sides, andit is mounted, for example, with a pitch of 9 degrees and roll of 13degrees.

Each lidar 310, 320 is operated to scan in essentially anomnidirectional radiation pattern, so they can see both forward andbackward.

Also shown are a primarily front-facing digital video camera 330 and aprimarily side and rear-facing digital video camera 340.

The lidars and cameras can be enclosed in a mirror housing 230 that alsoincludes the two mirrors 210, 220 (as shown), but may also be packagedin a housing separate from such a mirror housing. Housing 230 may alsoenclose or support other electrical or electronic components such asantennas 280. In either arrangement, the lidars 310, 320 and cameras330, 340 (that is, as with their associated housing) are physicallysupported by a lower 240 and/or upper 250 mounting arm extending fromthe cab. As FIG. 2 indicates, signal and power wires 260 for the sensors300 are preferably routed inside of the mounting arm 240. In oneembodiment, the wiring can then run from the mounting arm 240 into thecab 110 where the control electronics and control computers that executeautonomous or semi-autonomous control of the truck(s) are located.

In some embodiments the sensors 300 may not be mounted in or within themirrors themselves. What is important is that they are located on oroutboard of the left and right sidewalls of the cab, positioned outsideof the envelope of the truck 100. Being outside of the envelope is whatenables them to provide improved detection.

The location for the sensors 300 should be chosen such that othervehicles, objects and/or navigational landmarks that are forward, behindand to the side of the truck 100 are within a field of view of at leastone sensor. Vehicles may include other trucks, passenger cars, SportsUtility Vehicles (SUVs), motorcycles, and the like. Objects encounteredalong a road may include moving things such as people or animals, andnavigational landmarks may include features of the surrounding terrain,mountain peaks, utility poles, pavement edges, walls, fences, bridges,tunnels or anything else that is fixed in position and recognizable ordetectable,

Moving on to FIGS. 3A to 3D, the sensors 300 are located and operated ina particular way so that overlapping fields of view are provided on boththe left and right hand sides of the truck, as well as anotheroverlapping field of view in a direction forward of the truck. Thefields of view may be two-dimensional, scanning in a single plane, orthree-dimensional. FIG. 3A is an example where there are two,three-dimensional (3D) lidar fields of view (FOVs) on the vehicle'sright side, showing how the fields of view of the two lidars 310, 320overlap approximately at the middle and forward of the cab 110 such asin a region 370. More particularly, the right-side lidar 310-R mayprovide rearward-looking field of view 312-R, and right-side lidar 320-Rmay provide forward-looking field of view 322-R. In some embodiments,the exact amount of overlap 370 (as shown in FIG. 3B) is notparticularly critical, and what is important is to ensure there is nogap between 312-R and 322-R, the primary reason for the overlap being tomake sure that any objects adjacent the sides of the truck will not bein a sensor “blind spot”. In other embodiments, the overlapping fieldsof view function primarily to eliminate occluded regions, known as rangeshadows, or to provide more pixels on the area(s) of interest. It shouldalso be understood that the exact arrangement of sensors 300, theirlocation on or within the mirrors, their respective fields of view willdepend upon the exterior design features of truck 100 and especially thetractor 110.

The sensors may be packaged and sold as a mirror housing 230 assemblythat can be retrofit to an existing truck 100. Such mirror housingassemblies may have suites of sensors that are arranged in positions andwith orientations that are optimized for specific models of tractors110. For example, one model of retrofit mirror housing 230 may bedesigned for a Peterbuilt truck, and another may be particular to aVolvo truck.

Turning attention to the top view of FIG. 3B, it is also the case thatthe two forward facing sensors on the left and right side also haverespective scan patterns that overlap. One important use for thesesensor outputs 300, in a semi-automated driving application, is topermit a following vehicle to accurately detect the position of the rearof the leading vehicle. This will be further explained below. Thegreater the forward facing field of view overlap, the closer that thevehicle in front can be and still be seen in both views.

Having overlap(s) 370 in the forward direction is meaningful forproviding the best performance in an autonomous truck application. Theview looking forward should provide a clear and accurate picture of therear of the vehicle in front. If that front vehicle is quite far away,the following vehicle's ability to “follow the leader” depends on anaccurate measurement of that longitudinal spacing, and any lateraloffset of the front truck. And particularly for the lidars, to have twolidars that are each measuring the forward truck allows a more accuratemeasurement of where the forward truck is. This improved result occursbecause with two sensors on the following truck, there are twice as manyusable data points detected (e.g., with two lidars, there are twice asmany lidar measurements per second) that are reflected from the reardoor of the lead truck. For cameras, the large lateral distanceseparating the left from the right-side mirrors leads to much improvedresolution in computing the longitudinal distance (or rate of change ofdistance) to the front vehicle trailer based on the parallax caused bytheir different points of view.

For many decades, mirrors have been placed in a privileged position onthe exterior of a cab portion of the tractor 110, to enable the driverto see as much as possible around the vehicle. The left side mirror isplaced as close as possible to the driver's seat, as the driver sits inan elevated position above the hood of the engine. One advantage to theapproach described here is that the lidars 310, 320 (and/or othersensors 330, 340) are now also placed in a similar location enablingthem to see everything from only two vantage points on the side of thevehicle.

In other mobile robot applications, the “prime real estate” is oftenconsidered to be at the very top of a robot. However for a passengervehicle such as a car or a truck, or a military personnel carrier, thatlocation is often very contested—other devices such as GPS antennas orradio antennas or weapons also want to be there. Also, anomnidirectional lidar is also often placed on the roof of autonomouscars. In contrast to such roof-centered placement, placing lidars andother sensors elsewhere on a car, such as one on each of the fourcorners of the car body, is generally thought to increase cost andreduce visibility due to the lowered position of the corners relative tothe roof.

In the case of a semitrailer truck, however, the roof of the cab portionof the tractor 110 is usually lower than the roof of the trailer 120.Thus, if the sensors are placed on the roof of the tractor 110, thetrailer 120 is going to occlude the field of view, at least towards therear, and the tractor 110 itself may occlude the view downward.

Another consideration is that often the same company does not own thetractor 110 as owns the trailer 120, and it is important to be able toswap trailers 120 rapidly. Therefore, it is relatively impractical tointroduce any sort of specialized equipment to the trailer 120, on adedicated basis or otherwise. Mounting any of the sensors 300 on thetrailer 120 would also require some communication back and forth to thetractor 110, and those signals have to be connected, somehow, to thecomputers in the tractor 110. So it is simpler, if the sensors 300 canbe placed on the tractor 110 itself to the extent possible (and evenexclusively), as opposed to on the trailer 120.

Also important to consider, particularly for lidar, is that asemitrailer truck 100 is a segmented vehicle. Unlike a car, the tractor110 and the trailer 120 will yaw relative to each other when the vehicle100 follows curves in the road. If there is a mixture of sensors, someon the tractor 110, and some on the trailer 120, that yaw angle oftenneeds to be measured accurately to compensate for such positiondifferences.

Therefore, there are many reasons why using two positions, on the top ofeach of the left and right side mirrors, is advantageous.

FIGS. 3C and 3D, front looking 322 and rear looking 312 lidar fields ofview, show that they extend slightly upwards and downwards. This ispreferable because it cannot be assumed, for example, that even if theterrain is locally flat, the tractor 110 and trailer 120 will be alwaysperfectly level with respect to one another. The road may have anincline, or slightly varying camber, or follow terrain that has a steepdrop off to one side or the other of the trucks). It can also be usefulto sense tall objects to the side like bridges, utility poles, andtunnel walls in order to, for example, control lateral position or todetermine the location of the truck 100 in a global map in a manner thatmimics the capacity of the global positioning system (GPS) to do so.

It is also important for the forward 322 and rearward 312 fields of viewto include the locations of lane markings next to the truck(s) 100. Forexample, a forward-looking camera 320 alone may not provide enoughinformation to detect where a follower truck 100 is with respect to atravel lane, since that view may be occluded by the hood of the tractor110, or occluded by the trailer 120 of a leader truck 100, especiallywhen the two trucks are preferably following closely. Detection of lanemarkings can compensate for delays in video data acquisition andprocessing. Thus, sensor outputs 300 that also provide adownward-looking view of lane markings can enable improved estimates ofwhere the wheels are relative to the travel lane.

Extending the fields of view, as shown in FIGS. 3C and 3D, enables thesensors 312, 322 to better detect lane markings as well as otherfeatures of other vehicles. This may include extending the field of viewto provide a complete view of the rear doors of a companion vehiclebeing followed, or to detect when a companion vehicle is changing lanes,even when the vehicles are travelling along a road having varying slope(that is, “camber”) in either the lateral or longitudinal direction, orboth. Extending the field of view downward or upward also enablesdetection of lane markings and/or features of an adjacent vehicle morefully, regardless of attitude of the vehicle body with respect to itschassis (that is, regardless of its pitch or roll with respect to theroad surface).

FIGS. 3E and 3F illustrate where the vehicle is following a road thathas camber in the longitudinal direction (in the case of FIG. 3E) and/orlateral direction (in the case of FIG. 3F). As depicted, the body of thevehicle pitches and/or rolls with respect to the chassis, which in turnaffects where the sensors are pointed. The camber of the road may also,independently, affect the position of certain features within the fieldof view of the sensors. It may be desirable for certain features toremain in the field of view of the sensors, such as the top of thetrailer of a companion vehicle in front (as shown), or lane markings orjersey barriers or other objects at a “predetermined” distance away.Therefore the field of view of the sensors 312, 322 should be chosenaccordingly. Keeping certain features in view of the sensors depends on(a) where the field of view nominally points (b) any capacity tocompensate for the effects of body attitude with respect to the terrainunderneath it and the camber of the road in the lateral or longitudinaldirection. As shown in FIGS. 3E and 3F, this may be accomplished bysetting the sensor field of view to extend upward, above the roof of thevehicle. The field of view may also extend downward from an in-planedirection of a local tangent plane, where that tangent plane is locatedimmediately beneath the vehicle.

It should be understood that the sensors 312, 322 may have a fixed fieldof view that extends in one or more directions. However in otherimplementations, the sensors may be actively steered upward and/ordownward and/or to one side or the other by a controller thatcompensates for road camber and vehicle attitude. In either case, thefeatures of interest may now remain visible.

FIG. 4 illustrates how a leader truck 100-L and a follower truck 100-Fand how the field of view of the set of sensors 300 can be utilized todetect lane markings 400 on either side of the trucks 100-L, 100-F, aswell as any other vehicle that might be in a blind spot (such as the SUV410 in a lane to the right of the follower truck 100-F). Both forwardlooking sensors 320-L, 320-R on the follower 100-F also see the rear ofthe leader truck 100-F ahead, as well as a motorcycle 430 that's in thelane to its left (which may be in a blind spot for the leader truck100-L).

It should be understood that FIGS. 3A-3F and FIG. 4 are representativeof the fields of view 312-L, 312-R, 322-L, 322-R, and are notnecessarily to scale. The field of view finally realized will varydepending on the different models of lidars or cameras used.Furthermore, the use of sensors 300 with narrower fields of view mightmotivate a skilled artisan to add more sensors to achieve the desiredtotal field of view or degree of overlap. One example lidar sensor is aVelodyne VLP-16. Commercially available lidar sensors can actually seemuch further than what is shown here, such as 40, 50, or even 100 metersout, and digital video cameras 330, 340 can resolve small objectsequally far away. It is therefore understood that the exact fields ofview to accomplish the advantages described herein therefore depend onthe different model(s) of lidars chosen, their exact positions on orwithin the mirror housings, and other factors such as environmentalconditions. What is important is that the sensors are arranged andpositioned to minimize occlusion by the respective exterior bodycomponents of the autonomous truck (including for example the hood ofthe tractor) while also maximizing the area around the autonomous truckfor which obstacles are detected.

FIG. 4 is also presented in the context of a convoy where two trucks aretravelling together such that the follower 100-F is an instrumentedfollower 100-F and the leader 100-L has a human driver or other mastercontroller. The master controller may be responsible for, or shareresponsibility, to coordinate control of the leader and follower, suchas to execute a lane change operation in which both the leader 100-L andfollower 100-F change lanes together as companions. But the samearrangements of sensors 300 is just as valuable for an instrumentedleader situation (where the follower 100-F is controlling how the leader100-L behaves). So, it can be valuable for each of the follower 100-Fand leader 100-L vehicles to communicate their respective sensorinformation to the other companion truck in the convoy. The informationcan be passed between vehicles using wireless data subsystems such asvehicle-to-vehicle (V2V), cellular, or other digital radios. Theinformation is then displayed to the human driver or control computer toconsider when making decisions. In one scenario, where there is a humandriver in the lead truck 100-L, and the follower 100-F is autonomous,the driver of the lead vehicle 100-L must consider that it iseffectively three times longer when making lane change decisions. Butnow the autonomous control systems on the follower 100-F can now havethe capacity to veto a desire of the lead vehicle to change lanes, or tosend information to the leader 100-L about what's in sensor or mirrorblind spots of either truck (more generally, to assist by providing datathat's less visible to that lead 100-L truck driver, based on thesensors that are on the follower 100-F).

Returning attention to FIG. 4, consider a situation where the lead truck100-L may not be able to detect the presence of the SUV to the right ofthe follower 100-F. By having information from the sensors on thefollower 100-F, the leader 100-L is now aware of that SUV and may decideto delay a lane change operation. The question, is of course, not simplywhether the leader 100-L can change lanes, but rather whether bothtrucks can change lanes safely.

The sensor arrangement described herein thus permits a control mechanismon the decision-maker, whether it be the leader 100-L or the follower100-F (and regardless of whether that control mechanism be fullyautomated or involve a human decision-maker), to consider obstacles tothe left and right of both trucks 100-L, 100-F (including even smallvehicles such as motorcycles) no matter where they are. Such decisionsmay now also consider lane markings on both sides of both vehicles100-L, 100-F (and to distinguish whether the markings are solid ordotted, whether they are white or yellow, or transitioning from one typeto another) as needed for situation awareness.

Continuing to reference FIG. 4 in the context of convoying, a leadertruck 100-L in front is helping control the follower truck 100-F.However, that leader 100-L is also occluding what the sensors 300 on thefollower 100-F can see. But another advantage of placing the sensors 300out on each of the left and right mirror arms is that, at least withregard to occlusion of the lane markings 400, the leader 100-L cannotsimultaneously occlude both the left line markings and the right lanemarkings (see also FIG. 6A). And so, by having forward-facing cameras330-L, 330-R on both sides of the follower 100-F vehicle, it is nowguaranteed that the follower 100-F will always be able to see at leasteither the left lane marking with the left camera 330-L or the leftlidar, or the right lane marking with the right camera 330-R or theright lidar. Occlusion of the lane markings 400 by the leader 100-L,which is, in fact, guaranteed to happen during a lane change, is nolonger a concern. Viewing the lane markings 400 as far ahead as thoseadjacent to the lead vehicle 100-L is also intrinsically valuable interms of providing more accurate information on lane curvature.

FIG. 5 is an example of what the right-side, rear-facing camera 340-Rmight see. Here the camera 340-R sees a vehicle 510 in what wouldotherwise be a blind spot for a driver, as well as the lane markings.The rear wheels of the tractor 120 are almost directly down below thepart of the tractor 110 where the sensors are mounted (e.g., within oron the mirrors. Thus, the output of this camera 340-R (and/or thedownward section of the output of lidar 320-R) can be processed toprovide a nearly direct measurement of the follower 100-F vehicle's lanecenter offset based on observing the location of the lane markings 400.It can be computed from the number of camera pixels that span the offsetin the image from the present position of the lane edge multiplied bythe distance spanned per pixel. The lane center offset can then be usedto help the autonomous software keep the instrumented truck 100-L or100-F in its lane. The lack of such a direct view of the lane markingsadjacent the wheels forces prior art control system(s) to attempt toreconstruct a lane center offset error indication based on less relevantdata.

FIG. 5B is an example depicting the output of the camera 340-R (and/ordownward portion of the lidar 320-R's field of view showing a laneoffset. FIG. 5C is an ideal expected image when the truck is perfectlyaligned with the travel lane.

FIG. 5D is a flowchart describing a process of using these visualmeasurements of lane center offset of the follower 100-F vehicle and avisual description of the ideal image data. The principle leverages theconcept of visual servoing where, in general, the offset of importantfeatures (lane markings 400 in this case) from their ideal positions isused as an error signal that is presented as input to a controller. Moreparticularly, the process may include steps as follows:

-   -   Step 501: Extract and Locate Lane Features: The image data (such        as that in FIG. 5B) is processed to identify and then locate the        lane markings 400, if any, in the image.    -   Step 502: Compute Offset in Image: The offset is computed as the        difference between the present location of the lane markings 400        and their ideal location 411.    -   Step 503: Convert from Pixels to Meters: A model of how the        camera 330 forms images is inverted to convert the offset from        image coordinates to scene (real world) coordinates.

In other embodiments, the first two steps 501, 502 can instead beaccomplished by computing the correlation (or other similarity measure)of the present (FIG. 5B) and ideal (FIG. 5C) images and finding thelocation of the peak in correlation.

FIG. 6A shows an example view from the forward-facing camera 330-R ofthe follower 100-F, where the lane center offset of the leader 100-L isso large that lane markings 400-L on the left are occluded by the leader100-L, but the lane markings 400-R on the right are not.

It can be important for the follower 100-F to mimic the lane centeroffset of the leader 100-L if, for example, the leader 100-L is avoidingan object that the sensors 300 on the follower 100-F cannot yet see. Thesensor 300 configuration can also measure the lane center offset of theleader 100-L derived from either (or both) lane marking(s) 400-L, 400-Radjacent to the leader 100-L and visible ahead. FIG. 6B depicts such asituation, where either the left side offset, right side offset, or boththe left and right side offset may be detected. Visual measurement ofthe leader 100-L's lane center offset in this manner may assist withdecision making when the follower 100-F vehicle is intended to mimic thelane offset of the leader 100-L.

FIG. 6C is a flowchart describing an example of such visual measurementof lane center offset of the leader 100-L vehicle and a visualdescription of the image data. Again, the offset of important features(lane markings 400 and tire edges in this case) from their idealpositions is used as an error signal that is presented as input to acontroller. In one example the process is as follows (see FIG. 6D):

-   -   Step 601: Extract and Locate Lane and Tire Features: The image        data is processed to identify and then locate the lane markings        and tire edges, if any, in the image.    -   Step 602: Compute Offset in Image: The offset is computed, for        example, as the average of the left and right offsets.    -   Step 603: Convert from Pixels to Meters: A model of how the        camera forms images is inverted to convert the offset from image        coordinates to scene (real world) coordinates.

Note that lane markings near the wheels are not generally occluded buthalf the offset of one marking can be used instead in that rare case.

In other embodiments, the first two steps can instead be accomplished bycomputing the correlation (or other similarity measure) of the presentand an ideal image and finding the location of the peak in correlation.

Returning attention to FIG. 6A briefly, the same arrangement of sensorscan be used to estimate a distance between the follower 100-F and leader100-R. In some embodiments, it can be valuable to measure the rate ofchange of distance to a companion vehicle or even higher timederivatives of distance such as acceleration. As is well known in theart that such measurements can reduce reaction time in feedforwardcontrol and improve accuracy in predictive control.

FIG. 7 depicts a set of lidar data points 700 that might be detected onthe right side of a follower 100-F truck. From the diagram, it can beseen that a motorcycle 720 which would normally be completely invisibleto a driver using the mirrors alone (since the motorcycle is actuallybelow the window of the cab in tractor 110) is actually quite visible tothe lidar(s) 310, 320. This is indicated by the area 770 in the lidaroutput(s) that does not have data points, indicating that discrete lidarpoints have actually intercepted the motorcycle 720. Note too that thelidar also detects a small animal 740 on the side of the travel lanes aswell.

So, if the lead truck 100-L veers to its right, and the follower 100-Ftruck and the motorcycle veer to their left, that motorcycle might, infact, be occluded from the perspective of the lead driver by the trailerof the lead vehicle (the rear of which is depicted as the whiterectangle on the right side of FIG. 7). As such, a human being drivingthe lead truck 100-L also cannot see that motorcycle 720 (or the animal740), either, in this particular configuration of vehicles 100. And, ifin fact there is no driver in the follower 100-F vehicle, there is nohuman being who has any sense that the motorcycle 720 is present. Thusthe sensors 300 can be used to communicate to the driver (or othercontroller) of the lead vehicle 100-L that there is an object 720 or 740that it cannot see making it unsafe to change lanes at the moment.

FIG. 7 also depicts how other functions are enabled by the suite ofsensors 300 to sense other types of objects in or near the path oftravel. For example, objects to the side like guard rails, bridges,utility poles, or tunnel walls 780 can now also be utilized to furthercontrol lateral position or to determine the location of the truck in aglobal map.

Implementation Variations

The foregoing description of example embodiments illustrates anddescribes systems and methods for implementing novel arrangement andoperation of sensors in a vehicle. However, it is not intended to beexhaustive or limited to the precise form disclosed.

The embodiments described above may be implemented in many differentways. In some instances, the various “computers” and/or “controllers”are “data processors” or “embedded systems” that may be implemented by aone or more physical or virtual general purpose computers having acentral processor, memory, disk or other mass storage, communicationinterface(s), input/output (I/O) device(s), and other peripherals. Thegeneral purpose computer is transformed into the processors withimproved functionality, and executes the processes described above toprovide improved operations. The processors may operate, for example, byloading software instructions, and then executing the instructions tocarry out the functions described.

As is known in the art, such a computer may contain a system bus, wherea bus is a set of hardware wired connections used for data transferamong the components of a computer or processing system. The bus orbusses are shared conduit(s) that connect different elements of thecomputer system (e.g., processor, disk storage, memory, input/outputports, network ports, etc.) to enables the transfer of information. Oneor more central processor units are attached to the system bus andprovide for the execution of computer instructions. Also attached tosystem bus are typically I/O device interfaces for connecting variousinput and output devices (e.g., sensors, lidars, cameras, keyboards,touch displays, speakers, wireless radios etc.) to the computer. Networkinterface(s) allow the computer to connect to various other devices orsystems attached to a network. Memory provides volatile storage forcomputer software instructions and data used to implement an embodiment.Disk or other mass storage provides non-volatile storage for computersoftware instructions and data used to implement, for example, thevarious procedures described herein.

Certain portions may also be implemented as “logic” that performs one ormore of the stated functions. This logic may include hardware, such ashardwired logic circuits, an application-specific integrated circuit, afield programmable gate array, a microprocessor, software, firmware, ora combination thereof. Some or all of the logic may be stored in one ormore tangible non-transitory computer-readable storage media and mayinclude computer-executable instructions that may be executed by acomputer or data processing system. The computer-executable instructionsmay include instructions that implement one or more embodimentsdescribed herein. The tangible non-transitory computer-readable storagemedia may be volatile or non-volatile and may include, for example,flash memories, dynamic memories, removable disks, and non-removabledisks.

Embodiments may therefore typically be implemented in hardware,firmware, software, or any combination thereof.

In some implementations, the computers or controllers that execute theprocesses described above may be deployed in whole or in part in a cloudcomputing arrangement that makes available one or more physical and/orvirtual data processing machines via on-demand access to a network ofshared configurable computing resources (e.g., networks, servers,storage, applications, and services) that can be rapidly provisioned andreleased with minimal management effort or service provider interaction.

Furthermore, firmware, software, routines, or instructions may bedescribed herein as performing certain actions and/or functions. It alsoshould be understood that the block and flow diagrams may include moreor fewer elements, be arranged differently, or be representeddifferently. Therefore, it will be appreciated that such descriptionscontained herein are merely for convenience and that such actions infact result from computing devices, processors, controllers, or otherdevices executing the firmware, software, routines, instructions, etc.

While a series of steps has been described above with respect to theflow diagrams, the order of the steps may be modified in otherimplementations. In addition, the steps, operations, and steps may beperformed by additional or other modules or entities, which may becombined or separated to form other modules or entities. For example,while a series of steps has been described with regard to certainfigures, the order of the steps may be modified in other implementationsconsistent with the principles of the invention. Further, non-dependentsteps may be performed in parallel. Further, disclosed implementationsmay not be limited to any specific combination of hardware.

No element, act, or instruction used herein should be construed ascritical or essential to the disclosure unless explicitly described assuch. Also, as used herein, the article “a” is intended to include oneor more items. Where only one item is intended, the term “one” orsimilar language is used. Further, the phrase “based on” is intended tomean “based, at least in part, on” unless explicitly stated otherwise.

The above description contains several example embodiments. It should beunderstood that while a particular feature may have been disclosed abovewith respect to only one of several embodiments, that particular featuremay be combined with one or more other features of the other embodimentsas may be desired and advantageous for any given or particularapplication. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the innovations herein, and one skill in the art may now, inlight of the above description, recognize that many further combinationsand permutations are possible. Also, to the extent that the terms“includes,” and “including” and variants thereof are used in either thedetailed description or the claims, these terms are intended to beinclusive in a manner similar to the term “comprising”.

Accordingly, the subject matter covered by this patent is intended toembrace all such alterations, modifications, equivalents, and variationsthat fall within the spirit and scope of the claims that follow.

1. An apparatus comprising: a pair of assemblies, each comprising aplurality of perception sensors, the assemblies mounted to an exteriorof a truck, wherein each assembly is further configured such that: a. afirst assembly is located on a left side of the exterior of the truckand a second assembly is located on a right side of the exterior of thetruck; b. the first and second assemblies are disposed in a locationthat is outboard of a respective left side or right side of the exteriorof the truck; c. at least others of the perception sensors in eachassembly further comprise a forward facing sensor and a rearward facingsensor that have at least one region of overlapping field of view alongat least one side of the truck; and wherein at least some of the sensorshave a field of view that extends upward and downward from a plane thatis tangent to a surface of a road on which the truck is travelling. 2.The apparatus of claim 1 where the extent of a field of view of at leastone of the sensors contains at least one certain feature of interestregardless of the attitude of the vehicle or the camber of the road. 3.The apparatus of claim 1 additionally comprising: a control computerconfigured to actively control the field of view of one or more of thesensors so that the certain feature of interest remains visible.
 4. Theapparatus of claim 1 further wherein the perception sensors are furtherdisposed such that (i) lane markings adjacent to the truck, and (ii)lane markings adjacent nearby a companion truck that is located forwardof the truck are each within a field of view of at least one perceptionsensor; and additionally comprising: a control computer, configured forprocessing outputs of the sensors to detect lane markings adjacent thetruck and to thereby determine a lane center offset of the truck; andprocessing outputs of the sensors to detect lane markings adjacent thecompanion truck and to thereby determine a lane center offset of thecompanion truck.
 5. The apparatus of claim 4 wherein the controlcomputer is further for: using the determined lane center offset of thetruck and the determined lane center offset of the companion truck forfurther controlling the lane center offset of the truck so as to mimicthe lane center offset of the companion truck.
 6. The apparatus of claim1 wherein the perception sensors are also further disposed such that anyadjacent nearby vehicles, objects and/or navigational landmarks that areforward, behind or to the side of the truck are within a field of viewof at least one perception sensor regardless of the attitude of thetruck body relative to the road below it or camber of the road beingfollowed in the lateral or longitudinal direction.
 7. The apparatus ofclaim 1 wherein the perception sensors are one or more of LiDAR, camera,radar, or sonar sensors.
 8. The apparatus of claim 1 wherein eachassembly is further disposed within or adjacent to a respective leftside or right side mirror housing.
 9. The apparatus of claim 1 whereinthe control computer is additionally for processing outputs of one ormore perception sensors to detect whether the companion truck is stayingin or departing from its respective lane.
 10. The apparatus of claim 1wherein two or more perception sensors in each assembly are mounted suchthat one sensor is at least partially vertically aligned with anothersensor.
 11. The apparatus of claim 3 wherein one or more of theperception sensors are arranged to minimize occlusion by respectiveexterior body components of the truck.
 12. The apparatus of claim 1wherein controlling lane center offset of the truck is furthercoordinated with information received from the companion truck.
 13. Theapparatus of claim 1 wherein two or more of the perception sensors onthe truck are arranged to detect objects located in blind spots from aperspective of the companion truck.
 14. The apparatus of claim 1 whereinthe perception sensors include a plurality of lidar sensors that arefurther arranged such that a union of their fields of view, as comparedto each individual sensor, either (i) reduces occlusion of areas ofinterest or (ii) increases usable lidar data points around both thetruck and the companion truck.
 15. The apparatus of claim 1 wherein dataprovided by one or more perception sensors detects landmarks adjacent apath of travel that are further utilized by the control computer todetermine the lane center offset of the truck.
 16. The apparatus ofclaim 1 wherein data provided by one or more perception sensors is usedby the control computer to estimate and/or control a distance or achange in distance to a companion truck.
 17. The apparatus of claim 16wherein at least some of the perception sensors are cameras that arefurther arranged to estimate distance to the companion truck.
 18. Theapparatus of claim 4 wherein data from the perception sensors is used bythe control computer to detect objects that are located in blind spotsof the companion truck.
 19. The apparatus of claim 4 wherein theperception sensor outputs on the truck are further used by the controlcomputer to detect a landmark adjacent the companion truck.